Optimization methods for the insertion, protection, and detection of digital watermarks in digital data

ABSTRACT

Disclosed herein are methods and systems for encoding digital watermarks into content signals. Also disclosed are systems and methods for detecting and/or verifying digital watermarks in content signals. According to one embodiment, a system for encoding of digital watermark information includes: a window identifier for identifying a sample window in the signal; an interval calculator for determining a quantization interval of the sample window; and a sampler for normalizing the sample window to provide normalized samples. According to another embodiment, a system for pre-analyzing a digital signal for encoding at least one digital watermark using a digital filter is disclosed. According to another embodiment, a method for pre-analyzing a digital signal for encoding digital watermarks comprises: (1) providing a digital signal; (2) providing a digital filter to be applied to the digital signal; and (3) identifying an area of the digital signal that will be affected by the digital filter based on at least one measurable difference between the digital signal and a counterpart of the digital signal selected from the group consisting of the digital signal as transmitted, the digital signal as stored in a medium, and the digital signal as played backed. According to another embodiment, a method for encoding a watermark in a content signal includes the steps of (1) splitting a watermark bit stream; and (2) encoding at least half of the watermark bit stream in the content signal using inverted instances of the watermark bit stream. Other methods and systems for encoding/decoding digital watermarks are also disclosed.

This application is a continuation of application Ser. No. 11/497,822,filed Aug. 2, 2006, now U.S. Pat. No. 7,457,962, issued Nov. 25, 2008,which is a division of application Ser. No. 09/789,711, filed Feb. 22,2001, now U.S. Pat. No. 7,107,451, issued Sep. 12, 2006, which is acontinuation-in-part of patent application Ser. No. 09/281,279, filedMar. 30, 1999, now U.S. Pat. No. 6,522,767, issued Feb. 18, 2003, whichis a continuation of application Ser. No. 08/677,435, filed Jul. 2,1996, now U.S. Pat. No. 5,889,868, issued Mar. 30, 1999. The previouslyidentified patents and/or patent applications are hereby incorporated byreference, in their entireties.

This application is related to U.S. patent application Ser. No.08/674,726, filed Jul. 2, 1996, entitled “Exchange Mechanisms forDigital Information Packages with Bandwidth Securitization, MultichannelDigital Watermarks, and Key Management”; U.S. patent application Ser.No. 08/999,766, filed Jul. 23, 1997, entitled “Steganographic Method andDevice”; U.S. patent application Ser. No. 09/046,627, filed Mar. 24,1998, entitled “Method for Combining Transfer Function withPredetermined Key Creation” (issued as U.S. Pat. No. 6,598,162); U.S.patent application Ser. No. 09/053,628, filed Apr. 2, 1998, entitled“Multiple Transform Utilization and Application for Secure DigitalWatermarking” (issued as U.S. Pat. No. 6,205,249); U.S. patentapplication Ser. No. 09/281,279, filed Mar. 30, 1999, entitled“Optimization Methods for the Insertion, Protection, and Detection ofDigital Watermarks in Digital Data” (issued as U.S. Pat. No. 6,522,767);U.S. Provisional Application Ser. No. 60/169,274, filed Dec. 7, 1999,entitled “Systems, Methods And Devices For Trusted Transactions”; U.S.patent application Ser. No. 09/456,319, filed Dec. 8, 1999, entitled“Z-Transform Implementation of Digital Watermarks” (issued as U.S. Pat.No. 6,853,726); U.S. patent application Ser. No. 09/545,589, filed Apr.7, 2000, entitled “Method and System for Digital Watermarking” (issuedas U.S. Pat. No. 7,007,166); U.S. patent application Ser. No.09/594,719, filed Jun. 16, 2000, entitled “Utilizing Data Reduction inSteganographic and Cryptographic Systems” (which is acontinuation-in-part of International Application No. PCT/US00/06522,filed Mar. 14, 2000, which PCT application claimed priority to U.S.Provisional Application No. 60/125,990, filed Mar. 24, 1999) (issued asU.S. Pat. No. 7,123,718); International Application No. PCT/US00/21189,filed Aug. 4, 2000(which claims priority to U.S. Patent Application No.60/147,134, filed Aug. 4, 1999, and to U.S. Patent Application No.60/213,489, filed Jun. 23, 2000, both of which are entitled, “A SecurePersonal Content Server”), U.S. patent application Ser. No. 09/657,181,filed Sep. 7, 2000, entitled “Method And Device For Monitoring AndAnalyzing Signals”; U.S. Provisional Patent Application No. 60/234,199,filed Sep. 20, 2000, entitled “Improved Security Based on Subliminal andSupraliminal Channels For Data Objects” (issued as U.S. Pat. No.7,127,615); and U.S. patent application Ser. No. 09/671,739, filed Sep.29, 2001, entitled “Method And Device For Monitoring And AnalyzingSignals,” U.S. patent application Ser. No. 09/731,039 entitled “Systemand Method for Permitting Open Access to Data Objects and For SecuringData Within the Data Objects,” filed Dec. 7, 2000(issued as U.S. Pat.No. 7,177,429); and U.S. patent application Ser. No. 09/731,040 entitled“Systems, Methods and Devices for Trusted Transactions, ” filed Dec. 7,2000 (issued as U.S. Pat. No. 7,159,116). The previously identifiedpatents and/or patent applications are hereby incorporated by reference,in their entireties.

In addition, this application hereby incorporates by reference, as iffully stated herein, the disclosures of U.S. Pat.No.5,613,004“Steganographic Method and Device”; U.S. Pat. No. 5,745,569“Method for Stega-Cipher Protection of Computer Code”; U.S. Pat. No.5,889,868 “Optimization Methods for the Insertion, Protection, andDetection of Digital Watermarks in Digitized Data”; and U.S. Pat. No.6,078,664, entitled “Z-Transform Implementation of Digital Watermarks.”

BACKGROUND OF THE INVENTION

The present invention relates to digital watermarks.

Digital watermarks exist at a convergence point where creators andpublishers of digitized multimedia content demand localized, securedidentification and authentication of that content. Because existence ofpiracy is clearly a disincentive to the digital distribution ofcopyrighted works, establishment of responsibility for copies andderivative copies of such works is invaluable. In considering thevarious forms of multimedia content, whether “master,” stereo, NTSCvideo, audio tape or compact disc, tolerance of quality degradation willvary with individuals and affect the underlying commercial and aestheticvalue of the content. It is desirable to tie copyrights, ownershiprights, purchaser information or some combination of these and relateddata to the content in such a manner that the content must undergodamage, and therefore a reduction in value, with subsequent,unauthorized distribution of the content, whether it be commercial orotherwise.

Legal recognition and attitude shifts, which recognize the importance ofdigital watermarks as a necessary component of commercially distributedcontent (audio, video, game, etc.), will further the development ofacceptable parameters for the exchange of such content by the variousparties engaged in the commercial distribution of digital content. Theseparties may include artists, engineers, studios, Internet accessproviders, publishers, agents, on-line service providers, aggregators ofcontent for various forms of delivery, on-line retailers, individualsand parties that participate in the transfer of funds to arbitrate theactual delivery of content to intended parties.

Since the characteristics of digital recordings vary widely, it is aworthwhile goal to provide tools to describe an optimized envelope ofparameters for inserting, protecting and detecting digital watermarks ina given digitized sample (audio, video, virtual reality, etc.) stream.The optimization techniques described hereinafter make unauthorizedremoval of digital watermarks containing these parameters asignificantly costly operation in terms of the absolute given projectedeconomic gain from undetected commercial distribution. The optimizationtechniques, at the least, require significant damage to the contentsignal, as to make the unauthorized copy commercially worthless, if thedigital watermark is removed, absent the use of extremely expensivetools.

Presumably, the commercial value of some works will dictate some levelof piracy not detectable in practice and deemed “reasonable” by rightsholders given the overall economic return. For example, there willalways be fake $100 bills, LEVI'S.RTM. jeans, and Gucci.RTM. bags, giventhe sizes of the overall markets and potential economic returns forpirates in these markets--as there also will be unauthorized copies ofworks of music, operating systems (Windows.RTM. 95, etc.), video andfuture multimedia goods.

However, what differentiates the “digital marketplace” from the physicalmarketplace is the absence of any scheme that establishes responsibilityand trust in the authenticity of goods. For physical products,corporations and governments mark the goods and monitor manufacturingcapacity and sales to estimate loss from piracy. There also existreinforcing mechanisms, including legal, electronic, and informationalcampaigns to better educate consumers.

SUMMARY OF THE INVENTION

According to one embodiment of the present invention, a system forencoding of digital watermark information in a signal is disclosed. Thesystem includes a window identifier for identifying a sample window inthe signal; an interval calculator for determining a quantizationinterval of the sample window; and a sampler for normalizing the samplewindow to provide normalized samples. In general, the sample window thatis identified will have a maximum and a minimum. The quantizationinterval is used to quantize normalized window samples. The normalizedsamples conform to a limited range of values that are proportional toreal sample values and comprise a representation of the real samplevalues with a resolution higher than the real range of values. Thenormalized values may also be divided by the quantization interval intodistinct quantization levels.

According to another embodiment of the present invention, a system forpre-analyzing a digital signal for encoding at least one digitalwatermark using a digital filter is disclosed.

The system includes a processor for identifying an area of the digitalsignal that will be affected by the digital filter and an encoder forencoding the at least one digital watermark in the digital signal. Theencoder encodes the at least one digital watermark so as to avoid the atleast one area of the digital signal that will be affected by thedigital filter.

According to another embodiment of the present invention, a system forpreprocessing a watermark message is disclosed. The system includes apre-processor for determining an exact length of a watermark message asit will be encoded, and a key generator for generating a watermark keythat provides at least one unique bit for each bit comprising thewatermark message.

According to still another embodiment of the present invention, a systemfor encoding a watermark in a digital signal is disclosed. The systemincludes a generator for generating a plurality of watermarkpseudo-random key bits, and an encoder for encoding the watermark in thedigital signal using the watermark pseudo-random key bits andcharacteristics of the digital signal. The generator may be a non-lineargenerator, a chaotic generator, etc.

According to another embodiment of the present invention, a system forencoding a watermark in a digital signal is disclosed. The systemincludes a mapper for mapping pseudo-random key and processing stateinformation to effect an encode/decode map using a generator, and anencoder for encoding the watermark in the digital signal using theencode/decode map and characteristics of the digital signal. Thegenerator may be a non-linear generator, a chaotic generator, etc.

According to another embodiment of the present invention, a system forencoding watermarks is disclosed. The system includes an inverter forinverting at least one instance of the watermark bit stream, and anencoder for encoding at least one instance of the watermark using theinverted instance of the watermark bit stream.

According to another embodiment of the present invention, a system foranalyzing composite digitized signals for watermarks is disclosed. Thesystem includes a first receiver for receiving a composite signal; asecond receiver for receiving an unwatermarked sample signal; an alignerfor time aligning the unwatermarked sample signal with the compositesignal; an adjuster for gain adjusting the time aligned unwatermarkedsample signal to a corresponding segment of the composite signal,determined when the signals are time aligned; an estimator forestimating a pre-composite signal using the composite signal and thegain adjusted unwatermarked sample signal; an estimator for estimating awatermarked sample signal by subtracting the estimated pre-compositesignal from the composite signal; and a scanner for scanning theestimated watermarked sample signal for watermarks.

According to another embodiment of the present invention, a method forpre-analyzing a digital signal for encoding a plurality of digitalwatermarks using a digital filter is disclosed. The method includes thesteps of (1) providing a plurality of digital watermarks; (2)determining an encoding level; and encoding each of the plurality ofdigital watermarks at substantially the same encoding level.

According to another embodiment of the present invention, a method forpre-analyzing a digital signal for encoding digital watermarks using adigital filter is disclosed. The method includes the steps of (1)providing a digital signal; (2) providing a digital filter to be appliedto the digital signal; and (3) identifying an area of the digital signalthat will be affected by the digital filter based on at least onemeasurable difference between the digital signal and a counterpart ofthe digital signal selected from the group consisting of the digitalsignal as transmitted, the digital signal as stored in a medium, and thedigital signal as played backed.

According to another embodiment of the present invention, a method forencoding a watermark in a content signal is disclosed. The methodincludes the steps of (1) splitting a watermark bit stream; and (2)encoding at least half of the watermark bit stream in the content signalusing inverted instances of the watermark bit stream.

According to another embodiment of the present invention, a method forencoding at least one watermark in a content signal is disclosed. Themethod includes the steps of (1) predetermining a number of bits in thecontent signal to be encoded, based on at least one of a fixed lengthkey and signal characteristics of the content signal; and (2) encodingthe watermark in the predetermined bits.

According to another embodiment of the present invention, a method forencoding at least one watermark in a content signal is disclosed. Themethod includes the steps of (1) locating at least one noise-like signalfeature in the content signal; and (2) encoding the at least onewatermark in substantially the same location as the at least onenoise-like signal feature.

According to another embodiment of the present invention, a method forencoding at least one digital watermark in a content signal isdisclosed. The method includes the steps of (1) measuring a perceivedsignal-to-error ratio; and (2) encoding the at least one watermark in achannel bound by a minimum and maximum signal-to-error level for thecontent signal.

According to another embodiment of the present invention, a method fordigital watermark encode/decode comprises the steps of: (1) measuring aperceived signal-to-error ratio; and (2) encoding at least one watermarkin a signal feature that is bound by a minimum and maximumsignal-to-error level for the digital signal.

According to another embodiment of the present invention, a method fordecoding a digital watermark from a content signal is disclosed. Themethod includes the steps of (1) receiving a suspect digital signal tobe analyzed; (2) subjecting the digital signal to a time-basedalignment; (3) using the time-based alignment to align amplitude valuesin the suspect digital signal; and (4) decoding a digital watermark.

According to another embodiment of the present invention, a method forencoding watermarks in a content signal is disclosed. The methodincludes the steps of (I) identifying a plurality of signal features inthe content signal; and (2) inserting watermark data in the signalfeatures. The signal features may be identified from relationshipsbetween multiple sample windows in the content signal.

According to another embodiment of the present invention, a method fordecoding watermarks in a content signal is disclosed. The methodincludes the steps of (1) identifying a plurality of signal features inthe content signal; and (2) decoding watermark data from the signalfeatures. The signal features may be identified from relationshipsbetween multiple sample windows in the content signal.

According to another embodiment of the present invention, a method forpre-analyzing a digital signal for encoding digital watermarks using adigital filter is disclosed. The method includes the steps of (1)identifying at least one of a frequency and a time delimited area of thesignal that will be affected by the digital filter; and (2) encoding atleast one digital watermark so as to avoid the identified area.

According to another embodiment of the present invention, a method forpre-analyzing a digital signal for encoding digital watermarks using adigital filter is disclosed. The method includes the steps of (1)identifying at least one change to the digital signal that will beaffected by the digital filter; and (2) encoding at least one digitalwatermark so the watermark survives the changes introduced by thedigital filter.

According to another embodiment of the present invention, a method forguaranteeing watermark uniqueness is disclosed. The method includes thesteps of (1) providing a watermark; and (2) attaching a timestamp to thewatermark.

According to another embodiment of the present invention, a method forguaranteeing watermark uniqueness is disclosed. The method includes thesteps of (1) providing a watermark; and (2) attaching a useridentification dependent hash to the watermark.

According to another embodiment of the present invention, a method forguaranteeing watermark uniqueness is disclosed. The method includes thesteps of (1) providing a watermark; and attaching a message digest ofwatermark data to the watermark.

According to another embodiment of the present invention, a system fordigital watermark encode/decode operations comprises: (1) a CODECdatabase comprising a plurality of CODECs; and (2) a processor whichencodes at least one watermark using at least one CODECs from the CODECdatabase.

According to another embodiment of the present invention, a method fordigital watermark encode/decode is disclosed. The method includes thesteps of (1) receiving a digital signal stream; (2) using one or more ofa plurality of watermarking CODECs; and (3) encoding/decoding at leastone of a digital watermark and associating one or more of a plurality ofwatermarking CODECs with a predetermined key.

According to another embodiment of the present invention, an article ofmanufacture comprises: a receiver to receive a digital signal; adetector to detect at least two of a plurality of digital watermarkslocated within the digital signal; and a processor that enables contentsignal manipulation of the digital signal based on successful detectionof at least two of the plurality of digital watermarks. The article mayalso include a verification module which verifies at least one detectedwatermark.

According to another embodiment of the present invention, a method forpreprocessing a digital data signal to authorize a plurality of uniquedescendant copies of the digital data signal is disclosed. The methodincludes the steps of (1) providing a digital data signal;

(2) identifying candidate bits of the digital data signal that will bemanipulated during embedding; (3) generating a digital watermark messageto be embedded based on at least one predetermined criterion; and (4)embedding the digital watermark message in the digital data signal. Thecandidate bits may be identified through a psychoacoustic orpsychovisual model.

According to another embodiment of the present invention, a method forpreprocessing a digital data signal to authorize a plurality of uniquedescendant copies of the digital data signal is disclosed. The methodincludes the steps of (1) providing a digital data signal; (2)identifying candidate bits of the digital data signal that will bemanipulated during scrambling; (3) generating a scrambling key on atleast one predetermined criterion; and (4) scrambling the digital datasignal with the scrambling key. The candidate bits may be identifiedthrough a psychoacoustic or psychovisual model.

According to another embodiment of the present invention, a method forcreating a descendant copy of a digital data signal is disclosed. Themethod includes the steps of (1) obtaining a model for the digital datasignal; and (2) generating a watermark for the descendant copy of thedigital data signal based on at least one criterion. The criterion maybe, for example, a geographical territory, a transaction identification,an individual identification, a use limitation, a signal domain, etc.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The present invention relates to implementations of digital watermarksthat are optimally suited to particular transmission, distribution andstorage mediums given the nature of digitally sampled audio, video, andother multimedia works.

The present invention also relates to adapting watermark applicationparameters to the individual characteristics of a given digital samplestream.

The present invention additionally relates to the implementation ofdigital watermarks that are feature-based. That is, a system wherewatermark information is not carried in individual samples, but iscarried in the relationships between multiple samples, such as in awaveform shape. For example, in a manner similar to the way a US $100bill has copy protection features including ink type, paper stock,fiber, angles of artwork that distort in photocopier machines, insertedmagnetic strips, and composite art, the present invention envisionsnatural extensions for digital watermarks that may also separatefrequencies (color or audio), channels in 3D while utilizingdiscreteness in feature-based encoding only known to those withpseudo-random keys (i.e., cryptographic keys) or possibly tools toaccess such information, which may one day exist on a quantum level.

There are a number of hardware and software approaches in the prior artthat attempt to provide protection of multimedia content, includingencryption, cryptographic containers, cryptographic envelopes or“cryptolopes,” and trusted systems in general. None of these systemsplaces control of copy protection in the hands of the content creator asthe content is created, nor provides an economically feasible model forexchanging the content to be exchanged with identification data embeddedwithin the content.

Yet, given the existence of over 100 million personal computers and manymore non-copy-protected consumer electronic goods, copy protection seemsto belong within the signals. After all, the playing (i.e., using) ofthe content establishes its commercial value.

Generally, encryption and cryptographic containers serve copyrightholders as a means to protect data in transit between a publisher ordistributor and the purchaser of the data (i.e., a means of securing thedelivery of copyrighted material from one location to another by usingvariations of public key cryptography or other more centralizedcryptosystems).

Cryptolopes are suited specifically for copyrighted text that istime-sensitive, such as newspapers, where intellectual property rightsand origin data are made a permanent part of the file. For informationon public-key cryptosystems see U.S. Pat. No. 4,200,770 to Hellman etal., U.S. Pat. No. 4,218,582 to Hellman et al., U.S. Pat. No. 4,405,829to Rivest et al., and U.S. Pat. No. 4,424,414 to Hellman et al., whichpatents are incorporated herein by reference. Systems are proposed byIBM and Electronic Publishing Resources to accomplish cryptographiccontainer security.

Digitally-sampled copyrighted material, that is binary data on afundamental level, is a special case because of its long term valuecoupled with the ease and perfection of copying and transmission bygeneral purpose computing and telecommunications devices. In particular,in digitally-sampled material, there is no loss of quality in copies andno identifiable differences between one copy and any other subsequentcopy. For creators of content, distribution costs may be minimized withelectronic transmission of copyrighted works. Unfortunately, seekingsome form of informational or commercial return via electronic exchangeis ill-advised absent the use of digital watermarks to establishresponsibility for specific copies and unauthorized copying. Absentdigital watermarks, the unlikely instance of a market of trusted partieswho report any distribution or exchange of unauthorized copies of theprotected work must be relied upon for enforcement. Simply, contentcreators still cannot independently verify watermarks should they chooseto do so.

For a discussion of systems that are oriented around content-basedaddresses and directories, see U.S. Pat. No. 5,428,606 to Moskowitz,which patent is incorporated herein by reference.

In combining steganographic methods for insertion of informationidentifying the title, copyright holder, pricing, distribution path,licensed owner of a particular copy, or a myriad of other relatedinformation, with pseudo-random keys (which map insertion location ofthe information) similar to those used in cryptographic applications,randomly placed signals (digital watermarks) can be encoded as randomnoise in a content signal. Optimal planning of digital watermarkinsertion can be based on the inversion of optimal digital filters toestablish or map areas comprising a given content signal insertionenvelope. Taken further, planning operations will vary for differentdigitized content: audio, video, multimedia, virtual reality, etc.Optimization techniques for processes are described in related patents,including U.S. Pat. No. 5,613,004, entitled “Steganographic Method andDevice” and U.S. Pat. No. 5,822,432, entitled “Method for Human AssistedRandom Key Generation and Application for Digital Watermark System,”which patents are incorporated herein by reference.

Optimization processes must take into consideration the general art ofdigitization systems where sampling and quantizing are fundamentalphysical parameters. For instance, discrete time sampling has a naturallimit if packets of time are used, estimated at 1.times.10.sup.-42second. This provides a natural limit to the sampling operation. Also,since noise is preferable to distortion, quantizing will vary givendifferent storage mediums (magnetic, optical, etc.) or transmissionmediums (copper, fiber optic, satellite, etc.) for given digitizedsamples (audio, video, etc.). Reducing random bit error, quantizationerror, burst error, and the like is done for the singular goal ofpreserving quality in a given digitized sample. Theoretical perfecterror correction is not efficient, given the requirement of a hugeallocation of redundant data to detect and correct errors. In theabsence of such overhead, all error correction is still based on dataredundancy and requires the following operations: error detection tocheck data validity, error correction to replace erroneous data, anderror concealment to hide large errors or substitute data forinsufficient data correction. Even with perfect error correction, thegoal of a workable digital watermark system for the protection ofcopyrights would be to distribute copies that are less than perfect butnot perceivably different from the original. Ironically, in the presentdistribution of multimedia, this is the approach taken by contentcreators when faced with such distribution mechanisms as the Internet.As an example, for audio clips commercially exchanged on the World WideWeb (WWW), a part of the Internet, 8 bit sampled audio or audiodownsampled from 44.1 kHz (CD-quality), to 22 kHz and lower. Digitalfilters, however, are not ideal because of trade-offs betweenattenuation and time-domain response, but provide the engineer orsimilarly-trained individual with a set of decisions to make aboutmaximizing content quality with minimum data overhead and considerationof the ultimate delivery mechanism for the content (CDs, cabletelevision, satellite, audio tape, stereo amplifier, etc.).

For audio signals, and, more generally, for other frequency-basedcontent, such as video, one method of using digital filters is toinclude the use of an input filter to prevent frequency aliasing higherthan the so-called Nyquist frequencies. The Nyquist theorem specifiesthat the sampling frequency must be at least twice the highest signalfrequency of the sampled information (e.g., for the case of audio, humanperception of audio frequencies is in a range between 20 Hz and 20 kHz,such signals should be sampled at a frequency of at least 40 KHz).Without an input filter, aliases can still occur leaving an aliasedsignal in the original bandwidth that cannot be removed.

Even with anti-aliasing filters, quantization error can still cause lowlevel aliasing which may be removed with a dither technique. Dither is amethod of adding random noise to the signal, and is used to de-correlatequantization error from the signal while reducing the audibility of theremaining noise. Distortion may be removed, but at the cost of addingmore noise to the filtered output signal. An important effect is thesubsequent randomization of the quantization error while still leavingan envelope of an unremovable signaling band of noise. Thus, dither isdone at low signal levels, affecting only the least significant bits ofthe samples. Conversely, digital watermarks, which may take the form ofessentially randomly-mapped noise, are intended to be inserted intosamples of digitized content in a manner such as to maximize encodinglevels while minimizing any perceivable artifacts that would indicatetheir presence or allow for removal by filters, and without destroyingthe content signal. Further, digital watermarks should be inserted withprocesses that necessitate random searching in the content signal forwatermarks if an attacker lacks the keys. Attempts to over-encode noiseinto known watermarked signal locations to eliminate the informationsignal can be made difficult or impossible without damaging the contentsignal by relying on temporal encoding and randomization in thegeneration of keys during digital watermark insertion. As a result,although the watermark occupies only a small percentage of the signal,an attacker is forced to over-encode the entire signal at the highestencoding level, which creates audible artifacts.

The present invention relates to methods for obtaining more optimalmodels to design watermark systems that are tamper-resistant given thenumber and breadth of existent digitized sample options with differingfrequency and time components (audio, video, pictures, multimedia,virtual reality, etc.).

To accomplish these goals, the present invention maintains the highestquality of a given content signal as it was mastered, with itswatermarks suitably hidden, taking into account usage of digital filtersand error correction presently concerned solely with the quality ofcontent signals.

Additionally, where a watermark location is determined in a random orpseudo-random operation dependent on the creation of a pseudo-randomkey, as described in U.S. Pat. No. 5,613,004, and unlike other forms ofmanipulating digitized sample streams to improve quality or encode knownfrequency ranges, an engineer seeking to provide high levels ofprotection of copyrights, ownership, etc. is concerned with the size ofa given key, the size of the watermark message and the most suitablearea and method of insertion. Robustness is improved through highlyredundant error correction codes and interleaving, including codes knowngenerally as q-ary Bose-Chaudhuri-Hocquenghem (BCH) codes, a subset ofHamming coding operations, and codes combining error correction andinterleaving, such as the Cross-interleave Reed-Solomon Code. Using suchcodes to store watermark information in the signal increases the numberof changes required to obliterate a given watermark. Preprocessing thewatermark by considering error correction and the introduction of randomdata to make watermark discovery more difficult, prior to watermarking,will help determine sufficient key size. More generally, absolute keysize can be determined through preprocessing the message and the actualdigital watermark (a file including information regarding the copyrightowner, publisher, or some other party in the chain of exchange of thecontent) to compute the absolute encoded bit stream and limiting oradjusting the key size parameter to optimize the usage of key bits. Thenumber of bits in the primary key should match or exceed the number ofbits in the watermark message, to prevent redundant usage of key bits.Optimally, the number of bits in the primary key should exactly matchthe watermark size, since any extra bits are wasted computation.

Insertion of informational signals into content signals have beencontemplated. More detailed discussions are included in related patentsentitled “Steganographic Method and Device” and “Method for HumanAssisted Random Key Generation and Application for Digital WatermarkSystem.” The following discussion illustrates some previously disclosedsystems and their weaknesses.

Typically, previously disclosed systems lack emphasis or implementationof any pseudo-random operations to determine the insertion location, ormap, of information signals relating to the watermarks. Instead,previous implementations provide “copy protect” flags in obvious,apparent and easily removable locations. Further, previousimplementations do not emphasize the alteration of the content signalupon removal of the copy protection.

Standards for digital audio tape (DAT) prescribe insertion of data, suchas ISRC (Industry Standard Recording Codes) codes, title, and time insub-code according to the Serial Copy Management System (SCMS) toprevent multiple copying of the content. One time copying is permitted,however, and systems with AES3 connectors, which essentially overridecopy protection in the sub-code as implemented by SCMS, actually have nocopy limitations. The present invention provides improvement over thisimplementation with regard to the ability of unscrupulous users to loaddigital data into unprotected systems, such as general computingdevices, that may store the audio clip in a generalized file format tobe distributed over an on-line system for further duplication. Thesecurity of SCMS (Serial Copy Management System) can only exist as faras the support of similarly oriented hardware and the lack of attemptsby those skilled in the art to simply remove the subcode data inquestion.

Previous methods seek to protect content, but the shortcomings areapparent. U.S. Pat. No. 5,319,735 to Preuss et al. discusses a spreadspectrum method that would allow for over-encoding of the described,thus known, frequency range, and is severely limited in the amount ofdata that can be encoded—4.3 8-bit symbols per second. However, with thePreuss et al. method, randomization attacks will not result in audibleartifacts in the carrier signal, or degradation of the content as theinformation signal is in the subaudible range. It is important to notethe difference in application between spread spectrum in military fielduse for protection of real-time radio signals, and encoding informationinto static audio files. In the protection of real-time communications,spread spectrum has anti-jam features, since information is sent overseveral channels at once. Therefore, in order to jam the signal, one hasto jam all channels, including their own. In a static audio file,however, an attacker has practically unlimited time and processing powerto randomize each sub-channel in the signaling band without penalty tothemselves, so the anti-jam advantages of spread spectrum do not extendto this domain.

In a completely different implementation, U.S. Pat. No. 5,379,345 toGreenberg seeks enforcement of broadcast contracts using a spreadspectrum modulator to insert signals that are then confirmed by a spreadspectrum-capable receiver to establish the timing and length that agiven, marked advertisement is played. This information is measuredagainst a specific master of the underlying broadcast material. TheGreenberg patent does not ensure that real-time downloads of copyrightedcontent can be marked with identification information unless alldownload access points (PCs, modems, etc.), and upload points for thatmatter, have spread spectrum devices for monitoring.

Other methods include techniques similar to those disclosed in relatedpatents and patent applications cited above, but lack the pseudo-randomdimension of those patent applications for securing the location of thesignals inserted into the content. One implementation conducted byMichael Gerzon and Peter Craven, and described by Ken Pohlmann in theThird edition of Principles of Digital Audio, illustrates a technologycalled “buried data technique,” but does not address the importance ofrandomness in establishing the insertion locations of the informationalsignals in a given content signal, as no pseudo-random methods are usedas a basis for insertion. The overriding concern of the “buried datatechniques” appears to be to provide for a “known channel” to beinserted in such a manner as to leave little or no perceivable artifactsin the content signal while prescribing the exact location of theinformation (i.e., replacing the least significant bits (LSB) in a giveninformation signal). In Gerzon and Craven's example, a 20-bit signalgives way to 4-bits of LSBs for adding about 27 dB of noise to themusic. Per channel data insertion reached 176.4 kilobits per second perchannel, or 352.8 kbps with stereo channels. Similarly attempted datainsertion by the present inventors using random data insertion yieldedsimilar rates. The described techniques may be invaluable tomanufacturers seeking to support improvements in audio, video andmultimedia quality improvements. These include multiple audio channelsupport, surround sound, compressed information on dynamic range, or anycombination of these and similar data to improve quality. Unfortunately,this does little or nothing to protect the interests of copyrightholders from unscrupulous pirates, as they attempt to create unmarked,perfect copies of copyrighted works.

The present invention also relates to the “Steganographic Method andDevice” patent and the “Method for Human-Assisted Random Key Generationand Application for Digital Watermark System” patent, as well as U.S.Pat. No. 5,745,569 entitled “Method for Stega-Cipher Protection ofComputer Code” as mentioned above, specifically addressing the weaknessof inserting informational signals or digital watermarks into knownlocations or known frequency ranges, which are sub-audible. The presentinvention seeks to improve on the methods disclosed in these patentapplications and other methods by describing specific optimizationtechniques at the disposal of those skilled in the art. These techniquesprovide an a la carte method for rethinking error correction,interleaving, digital and analog filters, noise shaping, nonlinearrandom location mapping in digitized samples, hashing, or making uniqueindividual watermarks, localized noise signal mimic encoding to defeatnoise filtering over the entire sample stream, super audible spreadspectrum techniques, watermark inversion, preanalyzing watermark keynoise signatures, and derivative analysis of suspect samples againstoriginal masters to evaluate the existence of watermarks withstatistical techniques.

The goal of a digital watermark system is to insert a given informationsignal or signals in such a manner as to leave few or no artifacts inthe underlying content signal, while maximizing its encoding level andlocation sensitivity in the signal to force damage to the content signalwhen removal is attempted. The present invention establishes methods forestimating and utilizing parameters, given principles of thedigitization of multimedia content (audio, video, virtual reality,etc.), to create an optimized “envelope” for insertion of watermarks,and thus establish secured responsibility for digitally sampled content.The pseudo-random key that is generated is the only map to access theinformation signal while not compromising the quality of the content. Adigital watermark naturally resists attempts at removal because itexists as purely random or pseudo-random noise in a given digitizedsample. At the same time, inversion techniques and mimicking operations,as well as encoding signal features instead of given samples, can makethe removal of each and every unique encoded watermark in a givencontent signal economically infeasible (given the potential commercialreturns of the life of a given copyright) or impossible withoutsignificantly degrading the quality of the underlying, “protected”signal. Lacking this aesthetic quality, the marketability or commercialvalue of the copy is correspondingly reduced.

The present invention preserves quality of underlying content signals,while using methods for quantifying this quality to identify andhighlight advantageous locations for the insertion of digitalwatermarks.

The present invention integrates the watermark, an information signal,as closely as possible to the content signal, at a maximal level, toforce degradation of the content signal when attempts are made to removethe watermarks.

General methods for watermarking digitized content, as well as computercode, are described in related patents entitled “Steganographic Methodand Device” and entitled “Method for Stega-Cipher Protection of ComputerCode.” Recognizing the importance of perceptual encoding of watermarksby the authors and engineers who actually create content is addressed inpatent “Method for Human Assisted Random Key Generation and Applicationfor Digital Watermark System.”

The present invention describes methods of random noise creation giventhe necessary consequence of improving signal quality with digitizationtechniques. Additionally, methods are described for optimizingprojections of data redundancy and overhead in error correction methodsto better define and generate parameters by which a watermarking systemcan successfully create random keys and watermark messages thatsubsequently cannot be located and erased without possession of the keythat acts as the map for finding each encoded watermark. Thisdescription will provide the backdrop for establishing truly optimizedwatermark insertion including: use of nonlinear (chaotic) generators;error correction and data redundancy analysis to establish a system foroptimizing key and watermark message length; and more general issuesregarding desired quality relating to the importance of subjectingwatermarked content to different models when the content may bedistributed or sold in a number of prerecorded media formats ortransmitted via different electronic transmission systems; this includesthe use of perceptual coding; particularized methods such as noiseshaping; evaluating watermark noise signatures for predictability;localized noise function mimic encoding; encoding signal features;randomizing time to sample encoding of watermarks; and, finally, astatistical method for analyzing composite watermarked content against amaster sample content to allow watermark recovery. All of these featurescan be incorporated into specialized digital signal processingmicroprocessors to apply watermarks to nongeneralized computing devices,such as set-top boxes, video recorders that require time stamping orauthentication, digital versatile disc (DVD) machines and a multitude ofother mechanisms that play or record copyrighted content.

As discussed above, the Nyquist Theorem proves that bandlimited signalscan be sampled, stored, processed, transmitted, reconstructed, desampledor processed as discrete values. In order for the theorem to hold true,the sampling must be done at a frequency that is at least twice thefrequency of the highest signal frequency to be captured and reproduced.Aliasing will occur as a form of signal fold over, if the signalcontains components above the Nyquist frequency. To establish thehighest possible quality in a digital signal, aliasing is prevented bylow-pass filtering the input signal to a given digitization system by alow-pass or anti-aliasing filter. Any residue aliasing which may resultin signal distortion, relates to another area of signal quality control,namely, quantization error removal.

Quantization is required in a digitization system. Because of thecontinuous nature of an analog signal (amplitude vs. time), a quantizedsample of the signal is an imperfect estimate of the signal sample usedto encode it as a series of discrete integers. These numbers are merelyestimates of the true value of the signal amplitude. The differencebetween the true analog value at a discrete time and the quantizationvalue is the quantization error. The more bits allowed per sample, thegreater the accuracy of estimation; however, errors still always willoccur. It is the recurrent nature of quantization errors that providesan analogy with the location of digital watermarks.

Thus, methods for removal of quantization errors have relevance inmethods for determining the most secure locations for placement ofwatermarks to prevent the removal of such watermarks.

The highest fidelity in digital reproduction of a signal occurs atpoints where the analog signal converges with a given quantizationinterval. Where there is no such convergence, in varying degrees, thequantization error will be represented by the following range:

+Q/2 and −Q/2, where Q is the quantization interval. Indeed, describingmaximization of the quantization error and its ratio with the maximumsignal amplitude, as measured, will yield a signal-to-error ratio (S/E)which is closely related to the analog signal-to-noise ratio (S/N). Toestablish more precise boundaries for determining the S/E, with rootmean square (rms) quantization error E.sub.rms, and assuming a uniformprobability density function 1/Q (amplitude), the following describesthe error:E.sub.rms=Q/(12).sup.1/2

Signal to quantization error is expressed as:S/E=[S.sub.rms/E.sub.rms].sup.2=3/2(2.sup.2n)

Finally, in decibels (dB) and comparing 16-bit and 15-bit quantization:S/E(db)=10 log[(3/2).sup.1/2(2.sup.n)].sup.2=101log(3/2)+2.sup.nlog2(or “=20log[(3/2).sup.1/2(2.sup.n)]”)=6.02n+1.76

This explains the S/E ratio of 98 dB for 16-bit and 92 dB for 15-bitquantization. The 1.76 factor is established statistically as a resultof peak-to-rms ratio of a sinusoidal waveform, but the factor willdiffer if the signal waveform differs. In complex audio signals, anydistortion will exist as white noise across the audible range. Lowamplitude signals may alternatively suffer from distortion.

Quantization distortion is directly related with the original signal andis thus contained in the output signal. This being the case,implementation of so-called quality control of the signal may usedither. As discussed above, dither is a method of adding random noise tothe signal to de-correlate quantization error from the signal whilereducing the audibility of the remaining noise. Distortion may beremoved at the cost of adding more noise to the filtered output signal.An important effect is the subsequent randomization of the quantizationerror while still leaving an envelope of an unremovable signaling bandof noise. Dither, done at low signal levels, typically affects only theleast significant bits of the samples.

Use of linear and nonlinear quantization can affect the output signal,and this trade-off must be considered for a system of watermarksdesigned to determine acceptable quantization distortion to contain thedigital watermark. For audio systems, block linear quantizationimplementations may be chosen. However, block floating point andfloating point systems, non-uniform companding, adaptive deltamodulation, adaptive differential pulse-code modulation, and perceptualcoding schemes (which are oriented around the design of filters thatclosely match the actual perception of humans) appear to providealternative method implementations that would cause higher perceptiblenoise artifacts if filtering for watermarks was undertaken by pirates.The choice of method is related to the information overhead desired.

According to one aspect of the present invention, the envelope describedin the quantization equations above is suitable for preanalysis of adigitized sample to evaluate optimal locations for watermarks. Thepresent example is for audio, but corresponding applications fordigitization of video may be implemented using the quantization of colorand luminance.

The matter of dither complicates preanalysis of a sample evaluated fordigital watermarks. Therefore, the present invention also defines theoptimal envelope more closely given three types of dither (this exampleis for audio, others exist for video): triangular probability densityfunction (pdf), Gaussian pdf, and rectangular pdf Again, the purpose isto establish better boundaries for the random or pseudo-random insertionof a watermark in a region of a content signal that would represent anarea for biding watermarks in a manner most likely to cause damage tothe content signal if unauthorized searches or removal are undertaken.Dither makes removal of quantization error more economical through lowerdata overhead in a system by shifting the signal to decorrelate errorsfrom the underlying signal. When dither is used, the dither noise andsignal are quantized together to randomize the error. Dither which issubtractive requires removing the dither signal after requantization andcreates total error statistical independence. Subtractive dither alsoprovides further parameters for digital watermark insertion given theultimate removal of the dither signal before finalizing the productionof the content signal. With nonsubtractive dither, the dither signal ispermanently left in the content signal. Errors would not be independentbetween samples. For this reason, further analysis with the three typesof dither should reveal an acceptable dither signal without materiallyaffecting the signal quality.

Some proposed systems for implementing copyright protection intodigitally-sampled content predicate the natural occurrence of artifactsthat cannot be removed. Methods for creating a digital signature in theminimized error that is evident, as demonstrated by explanations ofdither, point out another significant improvement over the art in thesystem described in the present invention and its antecedents. Everyattempt is made to raise the error level of error from LSBs to a levelat which erasure necessarily leads to the degradation of the “protected”content signal. Furthermore, with such a system, pirates are forced tomake guesses, and then changes, at a high enough encoding level over amaximum amount of the content signal so as to cause signal degradation,because guessing naturally introduces error. Thus, dither affects thepresent invention's envelope by establishing a minimum encoding level.Any encoding done below the dither level might be erased by the dither.

One embodiment of the present invention may be viewed as the provisionof a random-super-level non-subtractive dither which containsinformation (the digital watermark).

To facilitate understanding how this does not cause audible artifacts,consider the meaning of such encoding in terms of the S/E ratio. In anormal 16-bit signal, there is a 98 dB S/E according to the equationS/E=6.02n+1.76. Consider that the encoding of watermark informationlooks like any other error, except it moves beyond the quantizationlevel, out of the LSBs. If the error is of a magnitude expressed inbits, for example, 8 bits, then at that moment, the signal effectivelydrops to 8 bits (16-8). This corresponds to a momentary drop in S/E,referred to herein as the momentary S/E. Yet, these errors arerelatively few and far between and therefore, since the signal isotherwise comprised of higher-bit samples, a “Perceived S/E” may bederived which is simply the weighted average of the samples using the“Pure S/E” (the samples without watermark information) and those withthe Momentary S/E. As a direct consequence, it may be observed that asthe watermark map becomes more sparse, the number of errors introducedin a given range is reduced, and the higher the perceived S/E. It alsohelps that the error is random, and so over time, appears as whitenoise, which is relatively unobtrusive. In general, it is observed thatas long as introduced errors leave resulting samples within an envelopein the sample window described by minimum and maximum values, beforeerror introduction, and the map is sufficiently sparse, the effects arenot perceived.

In addition, it is possible to obtain an even higher Perceived S/E byallowing the range of introduced errors to vary between a minimum andmaximum amount. This makes the weighted average S/E higher by reducingthe average introduced error level. Yet, someone trying to erase awatermark, assuming they knew the maximum level, would have to erase atthat level throughout the data, since they would not know how theintroduced level varies randomly, and would want to erase allwatermarks.

A watermarking cipher could perform this operation and may alsointroduce the further step of local dither (or other noise)significantly above the quantization amplitude on a window by windowbasis, randomly, to restrict total correlation between the watermarksignal and the probability that it remains independent between samples,similar to the use of subtractive dither implementations that are mostlyconcerned with the ultimate removal of the dither signal withrequantization. This ability could be used to accomplish signal doping,which adds a degree of random errors that do not contain watermarkinformation so as to prevent differential analysis of multiplewatermarked copies. Alternatively, it could be used to mimic a specificnoise function in a segment of the signal in order to defeat attempts tofilter a particular type of noise over the entire signal. By varyingthis function between watermarks, it may help ensure that any particularfilter is of little use over the whole signal. By applying severalfilters in series, it seems intuitive that the net results would besignificantly different from the original signal.

The discussion may be more appropriately introduced with perceptualcoding techniques, but a watermarking system could also defeat somedetection and correction with dither by inserting watermarks into signalfeatures, instead of signal samples. This would be equivalent to lookingfor signal characteristics, independent of the overall sample as itexists as a composite of a number of signals. Basically, instead ofencoding on a bit per sample basis, one might spread bits over severalsamples. The point of doing this is that filtering and convolutionoperations, like “flanging,” which definitely change individual sampleson a large scale, might leave intact enough of a recognizable overallsignal structure (the relationship between multiple samples) to preservethe watermark information. This may be done by measuring, generalizing,and altering features determined by the relationships between samples orfrequency bands. Because quantization is strictly an art ofapproximation, signal-to-error ratios, and thus the dynamic range of agiven system are determined.

The choice of eliminating quantization distortion at the expense ofleaving artifacts (not perceptible) is a permanent trade-off evident inall digitization systems which are necessarily based on approximation(the design goal of the present invention in preanalyzing a signal tomask the digital watermarks make imperceptibility possible). The highfidelity of duplication and thus subsequent ability to digitally orelectronically transmit the finished content (signal) is favored byconsumers and artists alike. Moreover, where there continues to be aquestion of approximating in quantization--digital watermark systemswill have a natural partner in seeking optimized envelopes in themultitude and variety of created digitized content.

Another aspect of optimizing the insertion of digital watermarks regardserror correction.

Highly redundant error codes and interleaving might create a bufferagainst burst errors introduced into digital watermarks throughrandomization attacks. A detailed description follows from the nature ofa digitization system—binary data can be corrected or concealed whenerrors exist. Random bit errors and burst errors differ in theiroccurrence: Random bit errors are error bits occurring in a randommanner, whereas burst errors may exist over large sequences of thebinary data comprising a digitized signal. Outside the scope of thepresent invention are errors caused by physical objects, such as dustand fingerprints, that contribute to the creation of dropouts aredifferent from the errors addressed herein.

Measuring error with bit-error ratio (BER), block error ratio (BLER) andburst-error length (BERL), however, provides the basis of errorcorrection. Redundancy of data is a focus of the present invention. Thisdata necessarily relies on existing data, the underlying content. Toefficiently describe optimal parameters for generating a cryptographickey and the digital watermark message discussion of error correction anderror concealment techniques is important.

Forms of error detection include one-bit parity, relying on themathematical ability to cast out numbers, for binary systems includingdigitization systems, such as 2. Remainders given odd or even results(parity) that are probabilistically determined to be errors in the data.For more appropriate error detection algorithms, such as CyclicRedundancy Check Code (CRCC), which are suited for the detection ofcommonly occurring burst error. Pohlmann (Principles of Digital Audio)notes the high accuracy of CRCC (99.99%) and the truth of the followingstatements given a k-bit data word with m bits of CRCC, a code word of nbits is formed (m=n−k):

burst errors less than or equal to m bits are always predictable.

the detection probability of burst errors of m+1 bits=1-2.sup.-m.

the detection probability of burst errors longer than m+1bits=1-2.sup.-m

random errors up to 3 consecutive bits long can be detected.

The medium of content delivery, however, provides the ultimate floor forCRCC design and the remainder of the error correction system.

Error correction techniques can be broken into three categories: methodsfor algebraic block codes, probabilistic methods for convolutionalcodes, and cross-interleave code where block codes are used in aconvolution structure. As previously discussed, the general class ofcodes that assist in pointing out the location of error are knowngenerally as Hamming codes, versus CRCC which is a linear block code.

What is important for establishing parameters for determining optimizederror coding in systems such as digital audio are more specificallyknown as Reed-Solomon Codes which are effective methods for correctingburst errors. Certain embodiments of the present invention presupposethe necessity of highly redundant error codes and interleaving, such asthat done in Cross Interleave Reed-Solomon Code, to counter burst errorstypically resulting from randomization attacks. More generally, certainembodiments of the present invention include the use of Hamming Codes of(n, n) to provide n−1 bit error detection and n−2 bit error correction.Further, a Hamming distance of n (or greater than n) is significantbecause of the nature of randomization attacks. Such an attack seeks torandomize the bits of the watermark message. A bit can be either 0 or 1,so any random change has a 50% chance of actually changing a bit fromwhat it was (50% is indicative of perfect randomness). Therefore, onemust assume that a good attack will change approximately half the bits(50%). A Hamming distance of n or greater, affords redundancy on a closepar with such randomization. In other words, even if half the bits arechanged, it would still be possible to recover the message.

Because interleaving and parity make data robust for error avoidance,certain embodiments of the present invention seek to perform timeinterleaving to randomly boost momentary S/E ratio and help preventremoving keys and watermarks that may be subsequently determined not tobe “errors.”

Given a particular digital content signal, parity, interleaving, delay,and cross-interleaving, used for error correction, should be taken intoaccount when preprocessing information to compute absolute sizerequirements of the encoded bit stream and limiting or adjusting keysize parameters to optimize and perhaps further randomize usage of keybits. In addition, these techniques minimize the impact of errors andare thus valuable in creating robust watermarks.

Uncorrected errors can be concealed in digital systems. Concealmentoffers a different dynamic to establish insertion parameters for thepresent invention. Error concealment techniques exist because it isgenerally more economical to hide some errors instead of requiringoverly expensive encoders and decoders and huge information overheads indigitization systems. Muting, interpolation, and methods for signalrestoration (removal of noise) relate to methods suggested by thepresent invention to invert some percentage or number of watermarks soas to ensure that at least some or as many as half of the watermarksmust still remain in the content signal to effectively eliminate theother half To invert a watermark relative to another watermark is toinverse the mathematical or logical relationships between the twowatermarks (for example, without limitation, by bit flipping, byinverting the phase relationships, or by using an inverse filterrelationship).

Given that a recording contains noise, whether due to watermarks or not,a restoration which “removes” such noise is likely to result in thechanging of some bit of the watermark message. Therefore, by invertingevery other watermark, it is possible to insure that the very act ofsuch corrections inverts enough watermark bits to create an inversewatermark. This inversion presupposes that the optimized watermarkinsertion is not truly optimal, given the will of a determined pirate toremove watermarks from particularly valuable content.

Ultimately, the inability to resell or openly trade unwatermarkedcontent will help enforce, as well as dictate, the necessity ofwatermarked content for legal transactions.

The mechanisms discussed above reach physical limits as the intent ofsignal filtering and error correction are ultimately determined to beeffective by humans—decidedly analog creatures. All output devices arethus also analog for playback.

The present invention allows for a preprocessed and preanalyzed signalstream and watermark data to be computed to describe an optimizedenvelope for the insertion of digital watermarks and creation of apseudorandom key, for a given digitized sample stream. Randomizing thetime variable in evaluating discrete sample frames of the content signalto introduce another aspect of randomization could further thesuccessful insertion of a watermark. More importantly, aspects ofperceptual coding are suitable for methods of digital watermarks orsuper-audible spread spectrum techniques that improve on the artdescribed by the Preuss et al. patent described above.

The basis for a perceptual coding system, for audio, ispsychoacoustics—the analysis of what the human ear is able to perceive.Similar analysis is conducted for video systems, and some may argueabused, with such approaches as “subliminal seduction” in advertisingcampaigns. Using the human for design goals is vastly different thandescribing mathematical or theoretical parameters for watermarks. Onsome level of digital watermark technology, the two approaches mayactually complement each other and provide for a truly optimized model.

The following example applies to audio applications. However, thisexample and other examples provided herein are relevant to video systemsas well as audio systems. Where a human ear can discern between energyinside and outside the “critical band,” (described by Harvey Fletcher)masking can be achieved. This is particularly important as quantizationnoise can be made imperceptible with perceptual coders given themaintenance of a sampling frequency, and decreased word length (data)based on signaling conditions. This contrasts with the necessarydecrease of 6 dB/bit with decreases in the sampling frequency asdescribed above in the explanation of the Nyquist Theorem. Indeed, dataquantity can be reduced by 75%. This is an extremely important variableto feed into the preprocessor that evaluates the signal in advance of“imprinting” the digital watermark.

In multichannel systems, such as MPEG-1, AC-3 and other compressionschemes, the data requirement (bits) is proportional to the square rootof the number of channels. What is accomplished is masking that isnonexistent perceptually, only acoustically. The phrase “nonexistentperceptually” means merely that the masking is not perceived as beingpresent.

Taken to another level for digital watermarking, which is necessary forcontent that may be compressed and decompressed, forward adaptiveallocation of bits and backward adaptive allocation provide for encodingsignals into content signals in such a manner that information can beconveyed in the transmission of a given content signal that issubsequently decoded to convey the relatively same audible signal to asignal that carries all of its bits—e.g., no perceptual differencesbetween two signals that differ in bit size. This coding technique mustalso be preanalyzed to determine the most likely sample bits, or signalcomponents, that will exist in the smaller sized signal. This is alsoclearly a means to remove digital watermarks placed into LSBs,especially when they do not contribute significant perceptible value tothe analyzed signal. Further methods for data reduction coding aresimilarly important for preanalyzing a given content signal prior towatermarking. Frequency domain coders, such as sub-band and transformbands, can achieve data reduction of ratios between 4:1 and 12:1. Thecoders adaptively quantize samples in each sub-band based on the maskingthreshold in that sub-band (See Pohlmann, Principles of Digital Audio).Transform coders, however, convert time domain samples into thefrequency domain for accomplishing lossless compression. Hybrid coderscombine both sub-band and transform coding, again with the ultimate goalof reducing the overall amount of data in a given content signal withoutloss of perceptible quality.

With digital watermarks, descriptive analysis of an information signalis important to preanalyze a given watermark's noise signature. Analysisof this signature versus the preanalysis of the target content signalfor optimized insertion location and key/message length, are potentiallyimportant components to the overall implementation of a securewatermark. It is important that the noise signature of a digitalwatermark be unpredictable without the pseudo-random key used to encodeit. Noise shaping, thus, has important applications in theimplementation of the present invention. In fact, adaptive dithersignals can be designed to correlate with a signal so as to mask theadditional noise—in this case a digital watermark. This relates to theabove discussion of buried data techniques and becomes independentlyimportant for digital watermark systems. Each instance of a watermark,where many are added to a given content signal given the size of thecontent and the size of the watermark message, can be “noise shaped” andthe binary description of the watermark signature may be made unique by“hashing” the data that comprises the watermark. Generally, hashing thewatermark prior to insertion is recommended to establish differencesbetween the data in each and every watermark “file.”

Additionally, the present invention provides a framework in which toanalyze a composite content signal that is suspected to contain awatermarked sample of a copyrighted work, against an unwatermarkedoriginal master of the same sample to determine if the composite contentactually contains a copy of a previously watermarked content signal.Such an analysis may be accomplished in the following scenario:

Assume the composite signal contains a watermark from the sample.

Assume the provision of the suspect composite signal C.sub.w(t) (wsubscript denotes a possible watermark) and the unwatermarked originalsample S.sub.uw(t). These are the only two recordings the analyzer islikely to have access to.

Now, it is necessary to recover a watermarked sample S.sub.w(t).

The methods of digital signal processing allow for the computation of anoptimal estimate of a signal. The signal to be estimated is thecomposite minus the watermarked sample, orC″.sub.w(t)=C.sub.w(t)-S.sub.w-(t). The analyzer, however, cannotdetermine a value of S.sub.w(t), since it does not know which of themany possible S.sub.w(t) signals was used in the composite. However, aclose estimate may be obtained by using S.sub.uw(t), since watermarkingmakes relatively minor changes to a signal.

So, C″.sub.w(t) (an estimate of C′.sub.w(t) given C″.sub.w(t) andS′.sub.w(t)) may be obtained. Once C″.sub.w(t) is calculated, it issimply subtracted from C.sub.w(t). This yields S′.sub.w(t)C.sub.w(t)-C″.sub.w(t). If the watermark is robust enough, and theestimate good enough, then S′.sub.w(t), which is approximately equal toS.sub.w(t), can be processed to extract the watermark. It is simply amatter of attempting watermark decoding against a set of likely encodingkey candidates.

Note that although a watermark is initially suspected to be present inthe composite, and the process as if it is, the specifics of thewatermark are not known, and a watermark is never introduced into thecalculations, so a watermark is extracted, it is valid, since it was notintroduced by the signal processing operations.

The usefulness of this type of operation is demonstrated in thefollowing scenario:

People are interested in simply proving that their copyrighted samplewas dubbed into another recording, not the specifics of ownership of thesample used in the dubbing. So, this implies that only a single, orlimited number of watermark keys would be used to mark samples, andhence, the decode key candidates are limited, since the same key wouldbe used to encode simple copyright information which never varies fromcopy to copy.

There are some problems to solve to accomplish this sort of processing.The sample in question is generally of shorter duration than thecomposite, and its amplitude may be different from the original.Analysis techniques could use a combination of human-assisted alignmentin the time domain, where graphical frequency analysis can indicate thetemporal location of a signal which closely matches that of the originalsample. In addition, automatic time warping algorithms which time alignseparate signals, on the assumption they are similar could also be usedto solve temporal problems. Finally, once temporal alignment isaccomplished, automatic amplitude adjustment could be performed on theoriginal sample to provide an optimal match between the compositesection containing the sample and the original sample.

It may be desirable to dynamically vary the encoding/decoding algorithmduring the course of encoding/decoding a signal stream with a givenwatermark. There are two reasons for dynamically varying theencoding/decoding algorithm.

The first reason for dynamically varying the encoding/decoding algorithmis that the characteristics of the signal stream may change between onelocality in the stream and another locality in the stream in a way thatsignificantly changes the effects that a given encoding algorithm mayhave on the perception of that section of the stream on playback. Inother words, one may want the encoding algorithm, and by implication,the decoding algorithm, to adapt to changes in the signal streamcharacteristics that cause relative changes in the effects of theencoding algorithm, so that the encoding process as a whole causes fewerartifacts, while maintaining a certain level of.security or encoding agiven amount of information.

The second reason for dynamically varying the encoding/decodingalgorithm is simply to make more difficult attempts at decodingwatermarks without keys. It is obviously a more difficult job to attemptsuch attacks if the encoding algorithm has been varied. This wouldrequire the attacker to guess the correct order in which to use variousdecoding algorithms.

In addition, other reasons for varying the encoding/decoding algorithmsmay arise in the future.

Two methods for varying of the encoding/decoding algorithms according toembodiments of the present invention are described herein. The firstmethod corresponded to adaptation to changing signal characteristics.This method requires a continuous analysis of the sample windowscomprising the signal stream as passed to the framework. Based on thesecharacteristics, which are mathematically well-defined functions of thesample stream (such as RMS energy, RMS/peak ratio, RMS differencebetween samples—which could reflect a measure of distortion), a newCODEC (encoder/decoder) module, from among a list of pre-defined CODECs,and the algorithms implemented in them, can be applied to the window inquestion. For the purpose of this discussion, windows are assumed to beequivalent to frames. And, in a frame-based system, this is astraightforward application of the architecture to provide automatedvariance of algorithms to encode and decode a single watermark. Thesignal features (or signal characteristics) can also be identified fromrelationships between multiple sample windows in the digital contentsignal. For example, an implementation using a sample window size of 15seconds can be compared to an implementation using a sample window sizeof 45 seconds. The comparison will reveal differences in robustness ofthe encoding and decoding operations—which differences will be useful inidentifying signal features that are desirable to target.

The second method for varying of the encoding/decoding algorithmscorresponds to increased security. This method is easier, since it doesnot require the relatively computationally-expensive process of furtheranalyzing the samples in a frame passed to the Framework. In thismethod, the Framework selects a new CODEC, from among a list ofpredefined CODECs, to which to pass the sample frame as a function ofthe pseudo-random key employed to encode/decode the watermark. Again,this is a straightforward application of framework architecture whichprovides automated variance of algorithms to encode and decode a singlewatermark versus limitations evident in the analysis of a single randomnoise signal inserted over the entire content signal as proposed byDigimarc, NEC, Thorn EMI and IBM under the general guise of spreadspectrum, embedded signaling schemes.

It is important to note that the modular framework architecture, inwhich various modules including CODECs are linked to keys, provides abasic method by which the user can manually accomplish such algorithmicvariations for independent watermarks. The main difference detailedabove is that an automated method to accomplish this can be used withinsingle watermarks.

Automated analysis of composited copyrighted material offers obviousadvantages over subjective “human listening” and “human viewing” methodscurrently used in copyright infringement cases pursued in the courts.

In addition to the embodiments discussed above, in the same manner thatthe signal to be watermarked, or scrambled, may be separated in time(from the beginning to the end of the signal) into streams, as disclosedin the copending U.S. patent application Ser. No. 09/594,719, entitled“Utilizing Data Reduction in Steganographic and Cryptographic Systems,”additional processing may occur prior to the actual embedding of thewatermark (“preprocessing”) as well as after each instance of watermarkembedding (“postprocessing”). The pre- and post-processing may be usedto optimize the actual embedding process.

In cases where the intent is to watermark a compressed file, for a givensignal, the coefficients that are to be manipulated may first beidentified. That is, for a particular signal, such as a song, a video,etc., to be embedded in such a manner as to authorize a plurality ofunique descendant copies of that signal, a preprocessing step may firstcomplete the necessary psychoacoustic or psychovisual modeling inherentin the embedding process for that signal. In one embodiment, thewatermark message may change for each descendant copy. In anotherembodiment, the key may change for each descendant copy to reflect theuniqueness of a given message. In either case, both the message and thekey may be processed so as to reflect any intended differences among thedescendant copies.

Of particular interest are those cases in which the psychoacoustic orpsychovisual model may be roughly correct for any number of proprietarycompression techniques (e.g., MP3, AAC, ePAC, etc. for audio, MPEG forvideo) and a given signal may need to be prepared so as to enable thewatermarking encoder to handle multiple requests for the same signal,but for different compression schemes. The preprocessing step of thepresent invention provides a “skeleton” of the candidate bits, andeliminates the need to process all of the signal data each time anencoding is performed. Each time a descendant copy is created, theappropriate model (e.g., psychovisual, psychoacoustic, etc.) may begenerated for that signal, or a previously saved version of theappropriate model is referenced, and then any concatenation between themodel and the particular compression scheme may be matched. Anyuniquely-generated descendant copy may differ from other copies in thatthe keys or messages may change to reflect the difference afterwatermark(s) have been embedded. One signal may have a plurality ofwatermarks, which may be a combination of different messages, differentencoding keys, and different decoding keys, or key pairs, for eachembedded message.

The psychoacoustic model itself may be saved for later handling in thewatermark encoding process so as to decrease the requisite time to makeevery instance of a descendant copy unique for any number ofcharacteristics. These characteristics may include, inter alia, ageographical territory, a transaction identification, an individualidentification, a use limitation, a domain, a logical constraint, etc.

The preprocessing of the signal may prepare a key for additionalpostprocessing in making the key or watermarked signal unique.

Other embodiments and uses of the invention will be apparent to thoseskilled in the art from consideration of the specification and practiceof the invention disclosed herein. The specification and examples shouldbe considered exemplary only with the true scope and spirit of theinvention indicated by the following claims. As will be easilyunderstood by those of ordinary skill in the art, variations andmodifications of each of the disclosed embodiments can be easily madewithin the scope of this invention as defined by the following claims.

1. A system for encoding of digital watermark information in a signal,comprising: at least one processor for processing data, said systemconfigured to use said processor in performing functions for followingelements: a window identifier for using time base alignment to identifysample windows in the signal to enable aligning watermark informationamplitude values with sample windows; an interval calculator fordetermining a quantization interval of the sample window, where thequantization interval is used to quantize normalized window samples; anda sampler for normalizing the sample window to provide normalizedsamples, where the normalized samples conform to a limited range ofvalues that are proportional to real sample values and comprise arepresentation of the real sample values with a resolution higher thanthe real range of values, and where the normalized values can be dividedby the quantization interval into distinct quantization levels.
 2. Thesystem of claim 1, further comprising: an analyzer for analyzing thenormalized samples to determine quantization levels; a comparator forcomparing a plurality of message bits in the signal to the correspondingquantization level information; an adjuster for adjusting thequantization level of the sample window to correspond to the message bitwhen a bit conflicts with the quantization level; and a denormalizer forde-normalizing the analyzed normalized samples.
 3. The system of claim1, further comprising: a quantization level analyzer for analyzing thequantization level of the samples to determine a message bit value. 4.The system of claim 1, further comprising: a processor for identifying aplurality of candidate bits in the signal that can be manipulated. 5.The system of claim 4, further comprising: an encoder for randomlyencoding watermark bits in the signal; and a filter; wherein fewer thana total number of available candidate bits are encoded as determined bythe filter.
 6. The system of claim 5, wherein the filter comprises atleast one of a primary mask, a convolution mask, and a message delimitermask.
 7. The system of claim 5, wherein the filter comprises an opticalfilter.
 8. The system of claim 5, wherein the filter comprises anoptimal digital filter.
 9. The system of claim 5, wherein the encoderencodes additional bits in the signal, which additional bits areunrelated to the watermark bits, and which additional bits serve tointroduce noise into the signal.
 10. The system of claim 5, furthercomprising: a decorrelator for decorrelating additional bits using adither technique.
 11. The system of claim 5, wherein the candidate bitsare selected from the group consisting of watermark bits and messagedelimiter bits.
 12. A method for digital watermark decode comprising: atleast one processor for performing the following steps: receiving asuspect digital signal to be analyzed; subjecting the digital signal toa time-based alignment; using the time-based alignment to alignamplitude values in the suspect digital signal; and decoding a digitalwatermark.
 13. A process of encoding digital watermark information in asignal, comprising: at least one processor for performing the followingsteps; identifying a sample window in the signal using a time basedalignment to enable aligning watermark information amplitude values withthe sample window; determining a quantization interval of the samplewindow, where the quantization interval is used to quantize normalizedwindow samples; and normalizing the sample window to provide normalizedsamples, where the normalized samples conform to a limited range ofvalues that are proportional to real sample values and comprise arepresentation of the real sample values with a resolution higher thanthe real range of values, and where the normalized values can be dividedby the quantization, interval into distinct quantization levels.
 14. Theprocess of claim 13, further comprising: analyzing the normalizedsamples to determine quantization levels; comparing a plurality ofmessage bits in the signal to the corresponding quantization levelinformation; adjusting the quantization level of the sample window tocorrespond to the message bit when a bit conflicts with the quantizationlevel; and de-normalizing the analyzed normalized samples.
 15. Theprocess of claim 13, further comprising: analyzing the quantizationlevel of the samples to determine a message bit value.
 16. The processof claim 13, further comprising: identifying a plurality of candidatebits in the signal.
 17. The process of claim 16, further comprising:encoding less than all of the identified candidate bits.
 18. The processof claim 16, wherein additional bits are encoded in the signalindependent of the identified candidate bits.
 19. A method of digitalwatermarking a signal comprising: at least one processor for performingthe following steps; identifying locations in the signal which locationsare suitable for encoding one or more watermark bits based on at leastone predetermined criterion; selecting a watermark message based on theidentification step; and encoding the watermark message using thelocations identified in the identification step.
 20. The method of claim19, wherein the step of identifying potential locations in the signalenables identification of candidate bits that can be manipulated.
 21. Adevice for digital watermarking comprising: a quantizer for quantizing asignal to estimate at least one envelope; an encoder for encoding atleast one watermark message in the estimated envelope; and a processorto dither the signal to estimate a minimum encoding level.
 22. Thedevice of claim 21, wherein the envelope is optimal.
 23. The device ofclaim 21, wherein the watermark message is associated with a watermarkmap.
 24. An article of manufacture comprising a non-transitory machinereadable medium, having thereon stored instructions adapted to beexecuted by a processor, which instructions when executed result in aprocess comprising: receiving a signal to be quantized; generating adither to estimate a minimum encoding level; and encoding at least onewatermark into the quantized signal using a watermarking cipher.